1 00:00:01,320 --> 00:00:10,770 ‫Extracting directory structure, crawling, so mapping the application layout and its structure is another 2 00:00:10,770 --> 00:00:11,870 ‫very important task. 3 00:00:12,800 --> 00:00:18,920 ‫Although there are some single page applications, the applications you will test can generally consist 4 00:00:18,920 --> 00:00:20,960 ‫of multiple Web pages. 5 00:00:21,950 --> 00:00:28,990 ‫And by multiple, I mean a lot in these pages can be independent or they can be linked to one another. 6 00:00:30,010 --> 00:00:36,580 ‫There's actually no magic to get the structure unless you're the actual developer of the application 7 00:00:37,060 --> 00:00:39,430 ‫and it is hours and hours of work. 8 00:00:41,960 --> 00:00:51,020 ‫So the best way to extract the structure of the application is to visit every page and click every link 9 00:00:51,200 --> 00:00:53,510 ‫and fill every form. 10 00:00:54,370 --> 00:01:03,160 ‫Then observe all the yourselves so you can manually walk through the application and identify Web pages 11 00:01:03,160 --> 00:01:07,690 ‫from authenticated and unauthenticated users perspective. 12 00:01:09,120 --> 00:01:14,670 ‫So this whole process is called crawling or spidering the application. 13 00:01:15,940 --> 00:01:21,580 ‫Now, you might think and you would be right that it's not really possible to do everything manually. 14 00:01:22,210 --> 00:01:24,520 ‫This really is a time consuming process. 15 00:01:25,570 --> 00:01:31,780 ‫So, gosh darn it, wouldn't you just love it if somebody created a tool for crawling, and how about 16 00:01:31,780 --> 00:01:33,280 ‫some scripts for good measure? 17 00:01:34,300 --> 00:01:36,040 ‫Well, I'm glad you asked. 18 00:01:37,060 --> 00:01:45,160 ‫Because now let's go over to Cali and open up your browser and berp, I know if you're raised like me, 19 00:01:46,000 --> 00:01:49,020 ‫it's rude, but I'm going to ask you to do it again. 20 00:01:49,510 --> 00:01:50,970 ‫Open your browser and burp. 21 00:01:51,460 --> 00:01:53,170 ‫OK, so sorry. 22 00:01:53,890 --> 00:01:57,370 ‫Go to Dashboard and make sure that capturing is active. 23 00:01:59,190 --> 00:02:02,190 ‫Then disable master interception from this button. 24 00:02:03,280 --> 00:02:04,510 ‫OK, so Skatoff. 25 00:02:05,410 --> 00:02:10,120 ‫But in your browser, redirect the traffic to berp by enabling Foxe proxy. 26 00:02:11,580 --> 00:02:17,190 ‫So now Berp will passively intercept all traffic coming from Firefox's. 27 00:02:18,800 --> 00:02:23,510 ‫And what we're going to do here is visit pages and click links. 28 00:02:25,190 --> 00:02:32,480 ‫So so that way, berp will create a structure of the application from the clicked you URLs by intercepting 29 00:02:32,480 --> 00:02:33,260 ‫passively. 30 00:02:35,070 --> 00:02:39,660 ‫Right, so I'm going to click on some links here on the page. 31 00:02:41,330 --> 00:02:42,410 ‫And then log in. 32 00:02:45,120 --> 00:02:47,610 ‫Open up a few other pages and. 33 00:02:50,210 --> 00:02:52,010 ‫OK, that's enough to show you. 34 00:02:53,790 --> 00:02:56,670 ‫So, OK, look at the history in BIR. 35 00:02:57,840 --> 00:02:59,700 ‫So all the requestor here. 36 00:03:01,310 --> 00:03:08,510 ‫And so the aim here is to see a sort of a broad overview of the application layout. 37 00:03:09,700 --> 00:03:10,180 ‫So. 38 00:03:11,890 --> 00:03:16,960 ‫There are a few more things in this step to perform, but I want you to just have a look here. 39 00:03:18,100 --> 00:03:20,650 ‫I'm sure that you've heard about robots that he. 40 00:03:23,000 --> 00:03:30,260 ‫I know almost every website uses this file to allow or disallow directories to be called by bots, so 41 00:03:30,260 --> 00:03:38,630 ‫robots dot text is a file that uses a specification protocol called robot exclusion protocol. 42 00:03:39,960 --> 00:03:45,540 ‫Now, it's not really something that you need to consider, but you might want to have a look at a later. 43 00:03:47,840 --> 00:03:55,310 ‫But, yeah, displaying this file is very handy if you want to see the sensitive pages and directories 44 00:03:55,310 --> 00:03:56,040 ‫easily. 45 00:03:56,840 --> 00:03:59,200 ‫So what am I saying? 46 00:03:59,420 --> 00:04:04,130 ‫Yeah, go to the robotics file of Boab. 47 00:04:05,640 --> 00:04:09,060 ‫And here are some directories that are not allowed. 48 00:04:09,900 --> 00:04:11,550 ‫So I'm going to visit each of them. 49 00:04:12,400 --> 00:04:13,420 ‫Admin directory. 50 00:04:14,780 --> 00:04:15,590 ‫Documents. 51 00:04:16,950 --> 00:04:22,200 ‫And images and then wait, what's this password's directory? 52 00:04:24,130 --> 00:04:25,060 ‫Now go to berp. 53 00:04:26,250 --> 00:04:27,630 ‫Click Target tab. 54 00:04:28,570 --> 00:04:29,560 ‫Quick site map. 55 00:04:31,030 --> 00:04:33,850 ‫On the left pane, you can see the site structure. 56 00:04:35,880 --> 00:04:43,710 ‫And also, there are filter options above, you know, this is great, I hope you get excited, as I 57 00:04:43,710 --> 00:04:50,790 ‫do so by clicking show all resources like 6000 and images, they're all going to show up on the map 58 00:04:50,790 --> 00:04:51,320 ‫as well. 59 00:04:53,800 --> 00:05:00,460 ‫I think the commercial version of Berp will do this task automatically, but I really do want you to, 60 00:05:01,360 --> 00:05:04,060 ‫you know, get your hands dirty, as I was saying before. 61 00:05:05,510 --> 00:05:12,710 ‫I know it's not the most efficient way, but this is how you learn, so we need something a bit more 62 00:05:12,710 --> 00:05:13,730 ‫intrusive. 63 00:05:13,850 --> 00:05:16,370 ‫Now, I should think so, Kelly. 64 00:05:16,370 --> 00:05:19,610 ‫Linux has a number of tools for this job. 65 00:05:20,980 --> 00:05:22,480 ‫There, Buster is one of them. 66 00:05:23,720 --> 00:05:28,130 ‫So open up the terminal and simply type there, buster. 67 00:05:29,440 --> 00:05:30,880 ‫And again, we will come up. 68 00:05:32,360 --> 00:05:37,130 ‫There, Buster, is the directory brute force here for Web applications. 69 00:05:39,100 --> 00:05:46,030 ‫So now let's provide the target, you, Earl, which is HTP Collins, so says one nine two one six eight 70 00:05:46,150 --> 00:05:49,660 ‫two zero for that one three zero BW slash. 71 00:05:50,990 --> 00:05:58,730 ‫And here's a starting point for door buster, and it will automate the tedious tasks of cataloging the 72 00:05:58,730 --> 00:06:00,650 ‫pages within the application. 73 00:06:01,220 --> 00:06:02,490 ‫That sounds good, huh? 74 00:06:03,800 --> 00:06:06,170 ‫So it works by requesting a Web page. 75 00:06:07,060 --> 00:06:14,470 ‫Parsing through it for links and then sending requests to these new links until all the Web pages are 76 00:06:14,470 --> 00:06:15,010 ‫mapped. 77 00:06:16,390 --> 00:06:20,440 ‫So then let's increase the number of threads here to 20. 78 00:06:21,660 --> 00:06:24,480 ‫And then choose a list, Brouse. 79 00:06:25,940 --> 00:06:29,780 ‫Now, there are several lists in Durban or directory under word lists. 80 00:06:31,240 --> 00:06:33,940 ‫And I'm going to choose the medium directory list. 81 00:06:35,450 --> 00:06:37,010 ‫OK, and QuickStart. 82 00:06:38,810 --> 00:06:46,820 ‫Oh, and another good thing is to identify administrative and test pages, these pages can contain sensitive 83 00:06:46,820 --> 00:06:52,220 ‫information and provide entry points to perform attacks such as a brute force attack. 84 00:06:53,340 --> 00:06:57,600 ‫And it's also possible to see old and backup files in the directory structure. 85 00:06:59,560 --> 00:07:03,970 ‫Don't laugh, I've seen it many times in real world situations. 86 00:07:04,870 --> 00:07:14,410 ‫So if the old version of the application functions and has any vulnerabilities, bingo, you can own 87 00:07:14,410 --> 00:07:15,580 ‫the entire system. 88 00:07:17,130 --> 00:07:22,960 ‫Besides that, the folders and files belong to the application, so there may be meta files and folders 89 00:07:23,290 --> 00:07:26,890 ‫of the server software as well as the application framework. 90 00:07:28,160 --> 00:07:32,930 ‫And I mean, no one files and folders such as my admin and so forth. 91 00:07:34,290 --> 00:07:36,770 ‫So Durbar here also looks for this kind of stuff. 92 00:07:37,940 --> 00:07:40,640 ‫And here, as you can see, it detects Drupal. 93 00:07:42,290 --> 00:07:44,420 ‫My admin and Escorial light. 94 00:07:45,780 --> 00:07:49,890 ‫OK, so at this point, I'm going to stop this scan and you can take a report. 95 00:07:51,760 --> 00:07:53,170 ‫Full text report. 96 00:07:54,240 --> 00:07:56,580 ‫Browse for the location, Issei and. 97 00:07:58,340 --> 00:07:59,150 ‫Give it the name. 98 00:08:00,020 --> 00:08:03,860 ‫I'm just going to type B rap and generate report. 99 00:08:06,820 --> 00:08:08,290 ‫So now you can go to that folder. 100 00:08:10,080 --> 00:08:12,090 ‫And here is a report we saved. 101 00:08:13,370 --> 00:08:14,570 ‫So now you can analyze it.